Pansombut, Tatdow. Advanced Learning Techniques for Improved Inference of Bayesian Belief Networks from Uncertain and High-dimensional Data. (under the Direction of Prof. Advanced Learning Techniques for Improved Inference of Bayesian Belief Networks from Uncertain and High-dimensional Data

PANSOMBUT, TATDOW. Advanced Learning Techniques for Improved Inference of Bayesian Belief Networks from Uncertain and High-dimensional Data. (Under the direction of Prof. Nagiza F. Samatova and Prof. Dennis R. Bahler.) A Bayesian Belief Network (BBN) is a powerful probabilistic learning model, it has been used successfully in many problem domains, such as medical diagnostics, computational biology, bioinformatics, image processing, and gaming. The success of Bayesian Belief Networks is due to a number of factors. A Bayesian Belief Network is capable of learning under uncertainty and can handle many types of data. It is also robust to model overfitting. However, the most powerful strength of a Bayesian Belief Network is, arguably, its ability to incorporate prior knowledge into the learning process; this characteristic is very unique to a Bayesian learning model. To learn a Bayesian Belief Network from data, there are two aspects of learning: (1) learning the network structure and (2) learning the conditional probabilities of variables in the network. Learning the conditional probabilities is relatively simple if we know the network structure; however, structure learning is an NP-hard problem. The reason is that the number of network structures grows super-exponentially in terms of the number of variables (features) in the network. Unfortunately, in some real world problem domains, it is not uncommon for a problem to have a large number of variables. Some of the reasons that contribute to a large number of variables in a problem are that (1) the problem is underdetermined, (2) the problem contains many irrelevant features, and (3) the problem contains some redundant features. In this research, we are interested in enhancing the learning ability of a Bayesian Belief Network by incorporating the knowledge obtained from other learning models. We improve (1) the BBN’s performance in terms of its prediction accuracy and computational learning time and (2) the BBN’s ability to learn complex and interesting relationships among its variables. We achieve these milestones through the following three complementary approaches that allow for uncovering some hidden knowledge about data using various supervised and unsupervised learning models and for using that knowledge to limit the search space for network structure inference. Component 1: Using a decision tree supervised learning model, we found an informative way to identify the lesser number of discriminatory features used as input for Bayesian Belief Network learning algorithms. As a result, we reduced the learning time by an order of magnitude and/or increased robustness and accuracy of a Bayesian Belief Network as a classifier. Component 2: Generalizing our method in Component 1 to highly underdetermined problems, we developed a method for constructing an ensemble of smaller-size BBN classifiers. This reduced the learning time and increased the prediction accuracy of a BBN classifier model for unconstrained problems. Component 3: For underdetermined problems with complex relationships between the data features and data samples, we designed a methodology for explicitly incorporating the knowledge learned about bicluster relationships into the ensemble of BBN classifiers. We were able to increase the prediction accuracy of a BBN classifier by learning complex relationships from the data that, otherwise, would not be identified when all examples were considered at once. When tested on real-world data sets in biology and climate, we consistently demonstrated the improved performance of our methods for hard-to-classify, highly unconstrained problems. © Copyright 2011 by Tatdow Pansombut